The dangers of testing without a documented test basis

If no documented test basis is available to the tester, there is a real risk that he or she will begin to rely on information sources other than the test basis, such as his or her intuition. An unwanted end result may be that system and documentation are running out of sync. If the system is correct and the documentation wrong, this can lead to maintenance or administration problems. Conversely, it is possible that (deep) functionality is described in the documentation that has been incorrectly implemented in the system and that has not emerged with testing based on sources other than the system documentation. Another unwanted end result may be that, in the absence of clarity concerning the scope, the testers generate an endless stream of change requests in the form of defects.

When to test solved defects

Defects that have been solved must be tested again. The timing of these tests can be quite different.

1. Test as soon as a defect is solved. The advantage of this is that the programmer, who has solved the defect, still has it fresh in his memory. He can therefore act quickly in the event that the defect appears not to be solved. The disadvantage is that the code is often changed, delivered and tested. Mistakes can be easily made here, and that is less efficient for the tester.
2. Gather solved defects and test these. The advantage of this is that defects can be solved and tested collectively (e.g. per module or per screen), which is a more effi cient way of working. The code is also more stable, so that the chances of a defect returning are minimal. The disadvantage, however, is that this method takes longer.

The choice of option 1 or 2 depends on the project and the way of working. If it is possible to deliver a release of an application every day1 and there are a large number of defects to be retested, the strategy may be to choose a mix of the above. It is then determined each day which solved defects will be included in the release and these can then be tested by the test team the following day. It is important in that case to set up a separate test environment and only to use it for testing the solved defects in the releases. In addition, a test of the entire test object will have to take place at the end, in order to establish that nothing else has changed (regression).